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ABSTRACT 


The  purpose  of  this  thesis  is  to  determine  if  format 
variation  has  any  affect  on  speed  and  accuracy  of  data 
entry  for  the  Marine  Integrated  Fire  and  Air  Support  System 
(MIFASS) . 

Independent  variables  for  the  test  are  message  formats, 
message  types,  and  subjects.  Subject  learning  effects  and 
experimenter  differences  are  controlled  by  use  of  a  pilot 
study.  Subjects  are  included  as  a  treatment  in  order  to 
estimate  their  effects. 

The  experiment  is  designed  to  support  both  parametric 
and  nonparametric  analyses.  A  4x5x8  randomized  complete 
block  factorial  with  two  replications  is  used  to  analyse 
the  effects  of  format  variation.  Friedman's  Two-Way  ANOV 
by  Ranks  is  used  to  test  subjective  rankings  of  the  formats 
as  concerns  ease  of  use  and  operational  suitability. 

It  is  concluded  that  format  variations  do  affect  speed 
of  data  entry  but  not  accuracy.  Format  3  appears  to  be 
superior.  Recommendations  for  further  study  are  given. 
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I.  STATEMENT  OF  THE  PROBLEM 


A.  THE  PROBLEM  IN  GENERAL 

The  primary  function  of  a  Fire  Support  Coordination 
Center  (FSCC)  at  any  level  of  command  is  to  provide  the 
commander  with  the  means  for  planning  and  coordinating 
available  fire  support.  Specific  responsibilities  of  a 
FSCC  include  fire  support  planning,  designation  of  targets 
and  assignment  of  priorities,  allocation  of  fire  support 
resources,  and  coordination  of  supporting  arms  with  each 
other  and  the  supported  force.  At  present,  all  these 
responsibilities  and  functions  are  carried  out  manually 
through  the  use  of  radio  and  telephone  communications, 
personal  liaison,  and  an  immense  amount  of  paperwork.  An 
inordinate  amount  of  valuable  time  is  lost  in  the  FSCC 
during  normal  operations  because  of  the  paperwork  and 
liaison  required  to  accomplish  its  mission.  In  an  attempt 
to  eliminate  this  bottleneck  and  more  effectively  use  the 
skill  and  training  of  the  personnel  who  operate  the  FSCC, 
the  Marine  Corp'  in  conjunction  with  Hughes  /-.ircraft 
Company,  is  developing  an  automated  system  wHch  vill 
materially  reduce  the  paperwork,  list  keeping,  and  manual 
labor  presently  required  in  a  FSCC.  The  system  will  be 
called  the  Marine  Intec  rated  Fire  and  Air  Support  System 
(MIFASS) . 
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B.  OVERVIEW  OF  THE  AUTOMATED  FSCC 

The  concept  for  MIFASS  operations  offers  extensive 
improvement  in  tactical  effectiveness  through  automation  of 
fire  request  planning,  fire  support  coordination,  fire  and 
air  support  planning,  conflict  prediction,  and  the  consoli¬ 
dation  of  many  other  common  functions  at  the  MIFASS  centers. 
The  automated  system  will  utilize  various  configurations  of 
the  hardware  listed  in  Table  I  at  designated  levels  of  the 
operational  tactical  unit.  Subunits  operating  under  tacti¬ 
cal  control  of  the  MIFASS  centers  will  input  intelligence 
data,  fire  support  requests,  med-evac  requests,  damage 
assessment  reports,  etc.,  to  che  MIFASS  computer.  MIFASS 
software  will  provide  rapid  response  to  many  on-line  users 
by  interleaving  the  processing  tasks  and  overlapping  the 
input/output  (I/O)  demands  with  processing.  Through 
generalized  data  management  software,  the  system  will  permit 
definition  of  new  messages,  files,  reports,  and  displays  in 
the  field  without  reprogramming.  The  application  programs 
envisioned  will  include  computer  assistance  and  algorithm 
development  for  major  requisite  functions  such  as  fire 
mission  analysis,  target  data  listing,  air  support  control, 
technical  fire  control,  conflict  detection,  troop  safety, 
mission  scheduling,  resource  allocation,  etc.  Since  each 
FSCC  will  have  access  to  the  memory  bank  and  coordinating 
agencies  will  be  able  to  view  the  same  data  simultaneously 
on  their  own  output  units,  the  time  needed  to  perform  the 
required  functions  for  all  subunits  will  be  materially 
decreased  thereby  increasing  the  effectiveness  of  the 
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overall  system,  Dae  to  the  probability  that  several  agencies 
will  be  viewing  the  same  data  simultaneously  and  in  the 
interests  of  simplification  in  training  and  programming/ 
it  is  evident  that  a  simplified  and  uniform  data  I/O  pro¬ 
cedure  will  be  required. 


TABLE  I 

HARDWARE  REQUIRED  FOR  A  FIELDED  MIFASS 

Each  FSCC  will  employ  a  specific  configuration  of  some  or 
all  of  these  units. 

Digital  Message  Entry  Device  (DMED) 

Portable  Printer 

Remote  Terminal  Device  (RTD) 

Cathode  Ray  Tube  (CRT)  Display 
Small  Auxiliary  Memory 
Hard  Copy  Printer 
Keyboard  Entry  Panel 
Communications  Net 

Computer  with  Internal  Memory  16-131K 

Auxiliary  Mass  Memory  500K-8M  Characters 

Magnetic  Tape  Units 

Printers 

Plotters 

Large  Screen  Displays 
Textual/Tabular  Displays 
Graphic  Displays 


C.  DATA  ENTRY  PROCEDURES 


Proper  data  I/O  procedures  are  crucial  to  the  function¬ 
ing  of  an  automated  system.  One  phase  of  these  procedures 
which  presents  a  potential  bottleneck  to  the  system  is  the 
format  used  when  entering  data  or  outputting  it  on  a  grid 
display  or  copier  for  viewing  by  interested  parties.  If  a 
uniform  format  is  not  used  by  everyone,  users  will  encounter 
problems  in  data  retrieval  and  in  interpretation  of  data 
entered  by  other  on-line  units.  The  requirement  for  a 
format  (or  formats)  understood  by  all  which  is  fast,  accu¬ 
rate,  and  convenient  is  obvious.  Determining  the  feasi¬ 
bility  of  such  a  format  was  the  overall  objective  of  this 
experiment. 

D.  PURPOSE  OF  THE  TEST 

As  one  approach  to  determining  the  feasibility  of  a  uni¬ 
form  format  for  use  in  the  system,  this  test  was  designed 
to  evaluate  differences  in  operator  data  entry  speed  and 
accuracy  resulting  from  the  use  of  alternative  message 
formats  when  used  with  differing  types  of  I/O  data.  The 
convenience  and  operational  suitability  of  the  formats 
tested  was  to  be  determined  through  operator  surveys. 

E.  HYPOTHESES  THAT  WERE  TESTED 

Hol:  T^ere  no  significant  difference  in  the  speed  of 
data  entry  due  to  the  use  of  different  message  formats. 

H^:  There  is  no  significant  difference  in  the  accuracy 
of  data  entry  due  to  the  use  of  different  message  formats. 
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Ho3{  There. is  no  significant  difference  in  the  speed  of 
data  entry  due  to  the  use  of  different  message  types. 

Hq^:  There  is  no  significant  difference  in  the  accuracy 
of  data  entry  due  to  the  use  of  different  message  types. 

Hq,.:  There  is  no  significant  difference  in  the  speed  of 
data  entry  due  to  the  subjects  used  for  the  test. 

HQg:  There  is  no  significant  difference  in  the  accuracy 
of  data  entry  due  to  the  subjects  used  for  the  test. 

Hq^:  There  is  no  significant  interaction  between  message 
type  and  format  for  speed  of  data  entry. 

Hgg!  There  is  no  significant  interaction  between  message 
type  and  format  for  accuracy  of  data  entry. 

Hog!  There  is  no  significant  interaction  between  message 
format  and  subject  for  speed  of  data  entry. 

Hq1q:  There  is  no  significant  interaction  between 
message  format  and  subject  for  accuracy  of  data  entry. 

Ho^^:  There  is  no  significant  interaction  between 
message  type  and  subject  for  speed  of  data  entry. 

Hq12i  There  is  no  significant  interaction  between 
message  type  and  subject  for  accuracy  of  data  entry. 

There  are  no  significant  interactions  among 
message  type,  message  format,  or  subject  for  speed  of  data 
entry. 

H0^^:  There  are  no  significant  interactions  among 
message  type,  message  format,  or  subject  for  accuracy  of 
data  entry. 

The  subjects  are  indifferent  to  the  message  for¬ 
mats  concerning  ease  of  use. 


13 


Ho16:  T*ie  su^jects  are  indifferent  to  the  message  for¬ 
mats  concerning  operational  suitability. 

P.  ASSUMPTIONS 

1.  Message  Types 

The  message  types  selected  for  this  experiment  were 
assumed  representative  of  the  total  population  of  messages 
envisioned  for  use  in  an  operational  MIFASS.  The  messages 
chosen  varied  in  length  from  62  to  573  characters  which  were 
a  mixture  of  numeric  and  alpha-numeric;  therefore,  they 
covered  a  wide  span  of  the  total  population. 

2.  Subjects 

The  subjects  used  for  this  test  were  assumed  repre¬ 
sentative  of  the  total  population  of  personnel  available  to 
operate  a  fielded  system.  This  assumption  might  be  ques¬ 
tioned  in  view  of  the  experience  and  proficiency  gained  by 
the  subjects  during  the  preceding  several  months  of  system 
development.  However,  circumstances  dictated  that  these 
particular  subjects  be  used,  and,  in  any  case,  it  is  a 
certainty  that  personnel  assigned  to  operate  a  fielded 
system  will  undergo  several  weeks  of  intensive  training 
before  being  permitted  to  begin  actual  operational  duties. 
Thus,  this  assumption  was  considered  justified. 

3.  Message  Format  Effects 

Effects  of  different  message  formats  on  the  speed 
and  accuracy  of  data  entry  under  test  conditions  were  con¬ 
sidered  typical  (ignoring  stress,  fatigue,  etc.)  of  the 
effects  which  will  be  prevalent  during  actual  operations. 
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If  it  is  subsequently  determined  that  any  conditions  make 
this  assumption  invalid,  this  experiment  can  be  repeated 
under  those  conditions  during  later  phases  of  MIFASS 
development. 


G.  GENERAL  APPROACH 

1.  The  Test 

Four  test  runs  were  made,  each  of  which  consisted  of 
a  group  of  subjects  entering  data  from  five  message  types 
into  a  specific  blank  message  format  which  was  presented  on 
a  grid  display  screen.  Each  subject  entered  data  twice  for 
each  message  type  for  a  total  of  10  trials  per  run  per 
subject.  Speed  and  accuracy  measures  were  taken  for  each 
trial.  Upon  completion  of  each  run,  the  subjects  each  com¬ 
pleted  a  subjective  evaluation  form  concerning  the  ease  of 
use  and  operational  suitability  of  that  format. 

2.  The  Analysis 

A  randomized  block  factorial  analysis  of  variance 
was  used  to  test  the  first  14  hypotheses.  The  two  subjec¬ 
tive  hypotheses  were  tested  using  nonparametric  test 
procedures . 

3.  Remarks 

It  was  not  the  objective  of  this  experiment  to  choose 
a  single  format  for  use  in  a  fielded  system  but  to  test  the 
feasibility  of  such  a  format.  However,  if  one  of  the  for¬ 
mats  tested  were  to  be  found  faster  and  more  accurate  than 
the  others  under  all  test  conditions,  it  might  be  assumed 


that  that  format  would  be  a  good  possibility  for  adoption 
as  a  universal  format  for  the  MIFASS. 
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II.  SELECTION  OF  DESIGN 


A.  DEPENDENT  VARIABLES 

1.  Speed  of  Data  Entry 

The  time,  T^,  required  to  enter  each  stimulus 
message  into  a  blank  message  format  was  collected  for  each 
subject  during  each  experimental  run.  This  response  was 
then  normalized  by  dividing  each  T^  by  the  total  number  of 
keying  actions  required  for  that  message  format  including 
any  header  data  or  spaces  over  which  the  operator  had  to 
space  or  tab  manually.  Thus,  the  dependent  variable  speed 
in  this  test  was  the  normalized  data  entry  time. 

2.  Accuracy  of  Data  Entry 

The  accuracy  of  data  entry  was  defined  in  terms  of 
data  elements  which  are  entire  sets  of  one  or  more  charac¬ 
ters  to  be  entered  into  a  single  message  field.  The  number, 
A^,  of  data  elements  that  had  been  entered  incorrectly 
was  collected  for  each  message  entered  by  each  subject  dur¬ 
ing  each  experimental  run.  Any  error  such  as  single  or 
multiple  typographical  errors  within  a  data  element,  an 
omitted  data  element,  or  one  data  element  substituted  for 
another  was  counted.  This  response  was  normalized  by 
dividing  A^  by  the  total  number  of  data  elements  that  were 
entered  in  the  i^  message.  Thus,  the  dependent  variable 
accuracy  in  this  test  was  the  normalized  data  entry  accuracy 
score. 
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3.  Subjective  Rankings 

To  forestall  the  possibility  that  individual  memories 
might  affect  the  ranking  of  the  formats  if  the  subjective 
opinions  were  to  be  gathered  after  all  the  formats  had  been 
presented,  it  was  considered  necessary  to  gather  a  separate 
opinion  from  each  subject  for  each  format  immediately  after 
it  had  been  tested.  A  graduated  interval  scale  was  designed 
for  each  subjective  dependent  variable.  Each  scale  con¬ 
sisted  of  20  intervals  with  the  endpoints  and  some  inter¬ 
mediate  intervals  marked  with  verbal  descriptions  for  ease 
in  marking  as  shown  in  Fig.  1.  These  scales  were  then  used 
for  gathering  the  individual  opinions. 

a.  Ease  of  Use 

The  graduated  scale  used  by  each  subject  to  give 
his  opinion  of  the  ease  of  use  of  each  format  was  marked 
from  0  to  20  where  0  coincided  with  the  rating  "Easy  to  Use" 
and  20  matched  "Unusable." 

b.  Operational  Suitability 

The  opinions  for  this  subjective  dependent 
variable  were  marked  on  a  graduated  scale  where  0  coincided 
with  "Very  Suitable"  and  20  with  "Very  Unsuitable." 

B.  INDEPENDENT  VARIABLES 
1.  Message  Formats 

The  independent  variable  of  primary  interest  was 
message  format.  There  were  four  different  formats  tested 
in  this  experiment.  Appendix  A  presents  the  actual  formats 
tested  for  each  message  type. 
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SUBJECT  REACTION  FORM 


Subject  _  Format  _ 

Experimenter  _  Date  _ 

Please  indicate  your  reaction  to  the  format  you  have  just 
completed  testing  by  placing  a  checkmark  at  the  appropriate 
place  on  the  line  below. 

1.  How  easy  or  difficult  was  it  to  use  this  format? 

0  5  10  15  20 

<  .  II  I| 4. 4  —I— I  ■  t ....I.,.,  a —I—  — 1— I— _1_ ..A .-1— i--  ..  I  —  I i— — L  -...I 

Easy  Convenient  Useable  May  be  Unuseable 

to  Use  Useable 


2.  How  would  you  rate  this  format  as  far  as  suitability 
for  use  in  an  operational  MIFASS? 

0  5  10  15  20 

*  *  »  ■  -  ■  »  .  i  i  t  i  i  t  i  ■  i  i  «  i  «  « 

Very  Suitable  Possibly  Unsuitable  Very 

Suitable  Suitable  Unsuitable 


Figure  1.  Subject  Reaction  Form.  This  form  was  used  to 
gather  data  necessary  to  the  nonparametric 
analyses  required. 


a.  Linear  Format 


In  this  format,  each  field  name  in  the  message 
was  immediately  followed,  on  the  same  line,  by  a  space  or 
spaces  in  which  the  pertinent  message  data  for  that  field 
was  to  be  entered.  One  field  followed  the  previous  field 
on  the  same  line  without  spacing  between  fields  until  that 
line  was  filled.  The  succeeding  fields  were  positioned  in 
lines  as  stated  until  the  entire  message  had  been  completed. 

b .  Block  Format 

The  block  format  had  the  field  names  in  a  linear 
sequence  with  the  message  data  space  or  spaces  immediately 
below  the  field  name  on  the  next  line  of  the  display  field. 
The  field  names  were  separated  sufficiently  to  preclude 
information  overlap  in  the  message  data  spaces  below.  No 
space  appeared  between  the  Jine  of  field  names  and  the  line 
containing  related  message  data.  A  double  space  appeared 
between  each  adjacent  pair  of  lines.  This  form  was  followed 
until  enough  line  pairs  appeared  to  complete  the  entire 
message. 

c.  Columnar/Separated  Format 

For  this  format,  all  field  names  appeared  in  a 
single  column  on  the  left  side  of  the  display  area.  The 
space  or  spaces  for  the  pertinent  message  data  for  each 
field  appeared  in  a  second  column  indented  from  the  left 
margin  of  the  display  area,  not  from  the  end  of  the  field 
name.  Thus,  the  message  data  column  appeared  as  a  distinct 
column  with  a  uniform  margin. 
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d.  Columnar/Run-On  Fermat 

This  format  had  the  field  names  in  a  column  as 
above,  but  the  space  or  spaces  for  message  data  appeared 
without  spacing  to  the  immediate  right  of  the  field  name. 
Instead  of  a  single  column  of-  field  names-message  data  as 
used  above,  three  columns  were  used  where  space  on  the 
display  area  existed;  otherwise,  two  columns  were  used. 

2.  Message  Types  5 

An  independent  variable  of  secondary  interest  was 
message  type.  As  previously  discussed,  the  five  types 
selected  for  this  experiment  were  assumed  to  be  representa¬ 
tive  of  the  total  population  of  messages  to  be  found  in  a 
fielded  system.  Data  pertinent  to  these  message  types  is 
shown  in  Table  II.  The  actual  messages  used  form  Appen¬ 
dix  B. 


TABLE  II 

DATA  ON  MESSAGE  TYPES 


Msg. 

Type 

Msg. 

Name 

Msg.  Name 

Number  of 
Characters 

Number  of 
Fields 

1 

CHKFIM 

Check  Fire  Command 

62 

6 

2 

EOMBDM 

End  of  Mission  Battle 
Damage  Assessment 

96 

10 

3 

TGTLST 

Target  List  Input 

151 

14 

4 

CFFTGR 

Call  for  Fire  by 

266 

23 

Target  Number 

FORDRM  Fire  Order  Universal 


5 


573 


54 


3.  Subjects 

The  other  independent  variable  of  secondary  interest 
was  the  subjects  participating  in  the  test.  As  discussed, 
these  subjects  were  assumed  to  be  representative  of  the 
total  population  of  potential  system  operators.  However, 
their  previous  training  and  experience  might  have  affected 
their  performance  and  might  have  introduced  bias  into  the 
experiment.  The  inclusion  of  subjects  as  treatments  per¬ 
mitted  an  analysis  of  their  effects  on  the  dependent  vari¬ 
ables  to  be  made.  Eight  subjects  were  used. 

4.  Replications 

The  independent  variable  of  tertiary  interest  was 
replication.  The  inclusion  of  this  variable  was  principally 
to  provide  a  check  to  see  if  the  experiment  was  conducted 
consistently. 

C.  CONTROLLED  VARIABLES 

There  were  several  nuisance  variables  which  had  the 
potential  to  bias  the  outcome  of  the  experiment.  Subjects 
were  such  a  potential  source  of  bias  and  their  effects  were 
controlled  by  including  them  as  a  treatment  in  the  experi¬ 
mental  design.  Other  nuisance  variables  could  not  be  exper¬ 
imentally  controlled  as  were  subjects  but  had  to  be  con¬ 
trolled  by  other  means. 

1.  Abbreviations  and  Acronyms 

This  source  of  bias  was  controlled  both  experimentally 
and  statistically.  The  use  of  abbreviations  and  acronyms  in 
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all  message  types  is  not  necessarily  proportional  to  the 
length  of  the  message;  therefore,  the  normalization  of  the 
dependent  variables  as  discussed  in  Para.  II. A  was  also 
utilized  to  statistically  control  this  effect  by  keeping 
the  variation  more  nearly  constant  over  the  normalized  data 
base.  Standard  abbreviations  and  acronyms  were  used  for 
all  message  types  for  all  formats.  This  assumption  held 
these  effects  constant. 

2.  Experimenter  Differences 

Two  experimental  controls  were  used  to  minimize 
effects  due  to  potential  experimenter  differences.  First, 
all  experimenters  were  trained  to  administer  the  experiment 
in  a  standard  fashion  thus  minimizing  effects  due  to  differ¬ 
ent  testing  procedures.  Second,  each  experimenter  presented 

each  format  twice  and  administered  no  more  than  one  test 

* 

run  to  any  given  subject. 

3.  Subject  Learning  Effects 

To  minimize  learning  effects,  two  experimental  con¬ 
trols  were  used:  training  and  randomization  of  presentation 
of  formats.  All  subjects  completed  the  Marine  Tactical 
Command  and  Control  System  (MTACCS)  Test  Bed  Operations 
Training  Course  prior  to  being  administered  the  test.  Such 
training  ensured  familiarity  with  all  five  message  types 
and  all  four  formats.  Thus,  when  proceeding  from  a  test 
run  with  one  format  to  the  next  run  with  another,  a  subject's 
learning  effects  were  minimized.  Remaining  effects  which 
could  possibly  introduce  bias  were  reduced  by  randomizing 


t 
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the  order  of  presentation  of  the  formats  and  message  types 
to  the  subjects. 

4.  Stimulus  Presentation 

a.  Message  Input  Stimuli 

The  stimulus  messages  were  presented  in  hard 
copy  form  only.  Radio  or  voice  presentation  was  deemed  too 
difficult  to  control  with  the  facilities  available  for  the 
test.  See  Appendix  B. 

b.  Format  Stimuli 

The  formats  used  for  this  test  were  presented 
to  the  subject  on  the  grid  display  screen.  One  format  was 
presented  at  a  time  in  the  order  shown  in  Table  III.  The 
random  ordering  of  the  presentation  of  each  format  minimized 
any  possible  effects  from  subject  learning  that  could  have 
occurred  if  the  formats  had  been  presented  to  all  the  sub¬ 
jects  in  the  same  order. 

D.  STATISTICAL  DESIGN 

Since  it  was  desired  to  secure  data  from  every  combin¬ 
ation  of  the  independent  variables,  a  randomized  complete 
block  factorial  design,  4><5x8  with  two  replications,  was 
selected  for  this  experiment.  This  design  required  320 
data  points  with  which  to  test  the  hypotheses  and  provided 
an  extremely  high  power  for  the  test.  Table  III  presents 
the  test  schedule  required  to  support  this  design. 
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TABLE  III  (Cont.) 


S1  S2  S3  S4  S5  S6  S7  S8 


Run 

4 


F  T  F 

T  F 

T  F 

T  F 

T  F 

1  £ 

T  F 

T 

5 

3 

1 

5 

5 

4 

3 

2 

3 

4 

2 

2 

1 

2 

2 

5 

4 

5 

5 

2 

3 

1 

3 

3 

2 

4 

5 

3 

3 

5 

5 

1 

4  l  4 

2  3 
5  J 

3  3 
2  J 

4  2 
5  * 

5  2 
1  ^ 

3  •L 

I* 

5 

4 

2 

3 

3 

4 

2 

5 

1 

4 

1 

1 

1 

1 

4 

4 

4 

1 

4 

2 

4 

1 

2 

2 

1 

3 

5 

1 

4 

3 

4 

1 

4 

2 

“2 

E1 

E4 

E3 

E2 

E1 

E4 

E. 

1.  Assumptions 
a.  Model 

The  mathematical  model  assumed  for  this  design 
was  a  fixed  factor  model  as  developed  in  Ostle  [Ref.  1] : 

Xi3kl  -  U+VVVV(AB>jk+<RC)jl+<BC)kl 

+<ABC)jkl+eijkl 

where  =  observation  ijkl 

mean  effect  (constant) 
effect  of  replicate  i,  i= 1 , 2 
effect  of  message  format  j,  j=l,2,3,4 
effect  of  message  type  k,  k=l,2,3#4,5 


xijki 

u 


Ri 

A. 
3 

B. 
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=  effect  of  subject  1,  1=1,2,. ..,8 
(AB) ^  etc.  are  interaction  terms 
e^jk^  =  random  error  for  observation  ijkl 

It  was  assumed  that  e^^i  was  distributed  normally  with  mean 
zero  and  variance  a  where  e  is  homoschedastic  for  all  i,j, 
k,  and  1.  It  was  further  assumed  that  there  were  no  repli¬ 
cation  by  treatment  interactions  and  that  the  observations 
were  independently  distributed  multivariate  normal. 

b.  Treatment  Effects 

As  discussed,  all  factors  in  this  model  were 
assumed  to  be  fixed.  There  is  no  doubt  cf  this  for  the 
formats  and  message  types;  however,  the  subject  sample 
might  be  argued  to  be  random  rather  than  fixed.  Consider¬ 
ing  the  experience  gained  by  all  the  subjects  through 
several  months  familiarization  with  the  test  bed  and  its 
equipment  and  their  overall  level  of  past  Marine  Corps 
operational  experience  and  training,  it  was  decided  that 
generalization  of  the  conclusions  made  from  this  test  over 
the  entire  population  of  possible  field  operators  would  not 
be  totally  valid.  Therefore,  of  necessity,  this  factor  had 
to  be  considered  fixed,  also. 

c.  Effects  of  Nonconformity  to  Assumptions 

In  general,  the  consequences  are  not  serious 
when  the  assumptions  made  in  connection  with  the  analysis 
of  variance  are  not  strictly  satisfied.  For  example,  minor 
deviations  from  normality  and/or  some  degree  of  heterosche- 
dasticity  will  have  little  effect  on  the  usual  tests  and 
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the  resulting  inferences.  Reference  2  discusses  this  aspect 
of  experimental  design  thoroughly. 

2.  Sample  Size 

To  determine  the  proper  sample  size  to  use  for  this 
test,  one  has  to  use  ideas  developed  by  Tang  [3]  and  used 
by  Pearson  and  Hartley  [4]  in  the  development  of  their 
power  function  curves.  Using  these  developments/  one  cal¬ 
culates  the  sample  size  required  to  yield  a  specified  power 
for  the  desired  test  utilizing  this  equation: 


J 

kDi2 

yj 

k 

o//n~ 

where  <j>  =  power  function 

D.  =  difference  between  the  effect  of  the  i^ 

1  treatment  and  the  mean  effect  of  all  treatments 

k  =  number  of  treatments  (i.e.,  number  of  levels 
of  the  treatment  in  question 

a  =  standard  deviation  of  errors 

n  =  number  of  observations  required  for  each 
treatment  level 

Since,  for  this  experiment,  a  and  were  not  known,  a  modi¬ 
fication  to  this  equation  was  made  to  circumvent  these 
problems.  It  was  assumed  that  a  maximum  detectable  treat¬ 
ment  difference  of  one  a  about  the  mean  would  be  sufficient 

2 

for  this  experiment.  It  can  be  shown  that  the  minimum 
occurs  when  two  of  the  treatment  differences  are  -  ^  and 
and  the  remaining  differences  are  0  [Ref.  7] .  Then 
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EiDi  =  (‘  l)  +(l)  +  0  +  •••  +  0  *  T 

Hence,  if  the  maximum  detectable  difference  is  other  than 
that  chosen,  the  power  of  the  test  will  increase.  Now,  the 
above  equation  for  4>  becomes 


This  form  was  much  simpler  to  utilize  and  was  no  longer 
sensitive  to  errors  in  estimating  a.  Using  this  power 
function  with  the  proper  degrees  of  freedom  and  a  =  0.05, 
the  available  power  can  be  determined  for  various  sample 
sizes  from  the  curves  produced  by  Pearson  and  Hartley  [4]. 
As  shown  in  Table  IV,  a  power  of  approximately  0.90  was 
attainable  for  the  test  of  format  differences  with  as  few 
as  three  subjects,  each  subject  using  two  samples  of  each 
message  type  for  each  format;  whereas,  four  subjects  were 
required  to  get  a  similar  power  for  the  test  of  message 
types.  Since  there  were  40  observations  for  each  subject, 
effects  of  subject  differences  could  be  tested  with 
extremely  high  power  for  almost  any  number  of  subjects. 
However,  as  the  use  of  eight  subjects  did  not  add  mater¬ 
ially  to  the  cost  or  time  required  but  did  enhance  the 
power  of  any  test  desired,  eight  subjects  were  used. 


TABLE  IV 


AVAILABLE 

POWERS  FOR  THE 

TEST 

No.  Of 
Subj . 

Obs./Subj ./ 
Cell 

Degrees  of 
Freedom  for 
Error  Term 

Power 

Format 

Msg. 

Type 

12 

2 

239 

0.999 

0.999 

10 

2 

199 

0.999 

0.999 

8 

2 

159 

0.999 

0.999 

6 

2 

119 

0.999 

0.982 

4 

2 

79 

0.972 

0.890 

4 

3 

158 

0.999 

0.984 

3 

2 

59 

0.895 

0.770 

3 

3 

118 

0.986 

0.950 

2 

2 

39 

0.710 

0.550 

2 

3 

78 

:  .910 

0.790 

2 

4 

117 

0.968 

0.900 
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III.  LOGISTICS  AND  TRAINING 


A.  SUBJECTS 

The  eight  subjects  used  for  this  test  were  Marine  Corps 
officer  and  enlisted  personnel  assigned  to  the  MTACCS  Test 
Bed  which  has  direct  responsibility  for  developing  MIFASS. 
Names  of  22  eligible  officers  and  men  formed  a  pool  from 
which  eight  were  drawn  randomly  to  act  as  subjects.  Pre¬ 
requisites  for  eligibility  were  previous  training  or 
experience  in  fire  support  coordination,  artillery  coordi¬ 
nation,  target  data  analysis,  and  test  bed  operations.  A 
capability  of  an  extremely  high  or  low  typing  speed  rendered 
potential  subjects  ineligible  as  it  was  desired  to  have 
typing  speed  on  a  level  representative  of  personnel  who  will 
serve  in  a  fielded  MIFASS.  Each  subject  had  completed  the 
MTACCS  Test  Bed  Operations  Course  which  covered  those 
subjects  listed  in  Table  V.  Prior  to  their  first  test  run, 
the  subjects  were  thoroughly  briefed  on  the  experiment. 

B.  EXPERIMENTERS 

The  four  experimenters  necessary  for  this  test  were 
selected  similarly  to  the  subjects  except  that  all  test  bed 
personnel,  including  the  civilian  contractor  employees, 
were  considered  eligible  for  duty  as  experimenters.  Avail¬ 
ability  was  the  prime  consideration  in  experimenter  selec¬ 
tion.  Those  selected  were  familiar  with  MTACCS  Test  Bed 
operations  through  the  above  mentioned  course  and  were  all 
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TABLE  VI 


HARDWARE  REQUIRED  FOR  THIS  EXPERIMENT 


Unit 

Model 

Quantity 

CDC  Computer  System 

3300 

1 

CDC  Grid  Display  System 

243-1 

1 

Call  Director 

600 

2 

Voice  Recorder 

TR-1720 

1 

Time  Code  Generator 

TC-201 

1 

D.  TEST  EQUIPMENT  AND  MATERIAL 

1.  Subject 

In  order  for  the  subjects  to  function  efficiently, 
they  had  at  their  disposal  on  the  grid  display  console  a 
reference  chart  and  a  MTACCS  Glossary.  The  reference  chart 
contained  field  header  data  and  allowable  field  data  entries 
used  in  all  MIFASS  messages  and  displays.  The  MTACCS 
Glossary  contained  all  the  abbreviations  and  acronyms  to  be 
used. 

2.  Experimenter 

Each  experimenter  had  at  his  disposal  the  following 
items:  a  stopwatch.  Parametric  Data  Collection  Forms, 

Subject  Reaction  Forms,  the  voice  recorder,  and  the  input 
message  stimuli. 


E.  STIMULUS  MATERIAL 


All  stimulus  materials  were  developed  by  Hughes  Aircraft 
Company  in  conjunction  with  Test  Bed  officers.  Message 
input  stimuli  for  the  five  messages  used  appear  in  App.  B. 
The  formats  presented  to  the  subjects  for  each  message 
appear  in  App.  A. 

F.  DATA  COLLECTION  FORMS 

To  support  the  analyses  necessary  for  this  test,  two 
data  collection  forms  were  necessary. 

1.  Parametric  Data  Collection  Form 

As  depicted  in  Fig.  2,  this  form  was  used  to  collect 
data  for  use  in  determining  speed  and  accuracy  measures. 

One  form  was  completed  by  the  applicable  experimenter  for 
each  subject  for  each  format.  A  total  of  32  forms  were 
required. 

2.  Subject  Reaction  Form 

As  discussed  earlier,  this  form  was  necessary  as  an 
intermediate  step  in  the  nonparametric  analysis.  A  total  of 
32  forms  were  required  as  each  subject  completed  one  for 
each  format  immediately  upon  completion  of  the  run  testing 
that  format.  Figure  1  depicts  this  form. 


Subject 

Format 


Date 


Experimenter 


Message  Type  Time  No.  of  Errors _ Remarks 


Experimenter  Notes 


Figure  2 .  Parametric  Data  Collection  Form. 

This  form  was  used  to  gather 
speed  and  accuracy  data  for  the 
ANOV. 
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IV.  TEST  PROCEDURE 

A.  PILOT  STUDY 

1.  Purpose 

The  pilot  study  was  primarily  conducted  in  order  to 
validate  the  test  design,  determine  if  the  desired  data 
could  be  collected,  and  see  if  the  required  analyses  could 
be  made  vising  this  data.  Secondary  considerations  were 
training  of  experimenters,  refinement  of  data  collection 
procedures  and  forms,  and  detection  and  correction  of  errors 
in  the  test  stimuli. 

2.  Administration 

a.  Materials 

The  test  stimuli,  equipment,  and  data  collection 
forms  used  were  those  to  be  used  during  the  actual  test. 

b .  Procedure 

The  test  administrator  selected  those  indivi¬ 
duals  to  be  used  as  experimenters  during  the  test.  Trial 
runs  were  made  with  each  selectee  acting  as  the  subject  to 
familiarize  him  with  the  entire  test.  Each  selectee  then 
performed  as  an  experimenter  for  other  runs  using  another 
eXj.'erimenter  as  a  subject.  Data  was  collected  as  it  was  to 
be  collected  during  the  actual  test.  This  procedure  was 
followed  until  the  test  administrator  felt  that  the  level 
of  experience  was  great  enough  to  minimize  any  effects  due 
to  experimenter  differences. 
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c.  Results 

A  trial  analysis  was  made  using  the  data 
gathered  during  the  pilot  study.  The  test  design  was 
confirmed  as  was  the  suitability  of  the  data  collection 
forms. 

B.  TEST  PROCEDURE 

The  actual  test  runs  were  conducted  as  follows: 

Prior  to  each  run,  the  test  administrator  ensured 
that  the  system  was  operating  properly  and  that  the  correct 
software  was  loaded. 

The  experimenter  presented  the  subject  with  the 
proper  message  stimuli  and  recorded  the  event  on  the  voice 
recorder. 

The  format  to  be  filled  in  by  the  subject  was  called 
to  the  screen  from  the  memory. 

When  the  format  to  be  tested  appeared ,  the  experi¬ 
menter  recorded  the  event  and  started  his  stopwatch. 

The  subject  entered  the  necessary  data  using  the 
alphanumeric  keyboard  on  the  grid  console. 

Upon  completion  of  his  task,  the  subject  pressed 
the  STORE  RECORD  key.  The  experimenter  stopped  his  stop¬ 
watch  and  recorded  the  time  on  the  voice  recorder  and  t.ie 
Parametric  Data  Collection  Form. 

The  above  steps  were  repeated  until  that  run  was 
complete.  The  subject  then  completed  his  Subject  Reaction 
Form, 
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C.  SUBJECT  BRIEFING 

Prior  to  commencement  of  the  first  test  run-  a  thorough 
briefing  was  given  to  all  subjects  and  experimenters.  The 
briefing  served  to  acquaint  the  subjects  with  the  test,  its 
purpose,  and  its  procedures.  The  test  administrator  pre- 
viewed  the  test,  answered  any  questions  the  subjects  or 
experimenters  had,  and  provided  each  subject  with  the  test 
schedule.  The  subjects  were  cautioned  not  to  discuss  their 
reactions  with  other  subjects  until  after  completion  of  the 
entire  test. 
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V.  ANALYSIS 


A.  PARAMETRIC  ANALYSIS  OF  VARIANCE  (ANOV) 

1.  Procedure 

a.  Data  Collection 

Test  data  for  the  dependent  variables  speed  and 
accuracy  were  collected  on  the  Parametric  Data  Collection 
Form,  The  time  required  to  enter  each  message  stimulus  into 
each  format  by  each  subject  was  recorded  by  the  experimenter 
but  the  accuracy  observations  were  collected  differently. 
Errors  were  detected  through  the  use  of  a  compare  program 
by  which  the  computer  compared  actual  entries  with  the  cor¬ 
rect  entries  and  provided  -i  printout  of  all  entries  con¬ 
taining  errors.  The  Test  Administrator  then  manually 
counted  and  recorded  these  errors.  The  data  collected  for 
both  parametric  analyses  is  presented  in  App.  C. 

b.  Normalization  of  Data 

In  order  to  preclude  effects  due  to  the  differ¬ 
ing  lengths  of  the  message  types,  the  test  data  had  to  be 
normalized. 

(1)  Speed.  The  normalizing  factors  for  speed 
of  data  entry  were  the  reciprocals  of  the  number  of  Keying 
actions  necessary  to  enter  a  message  type  into  a  particular 
format.  For  example,  for  Format  1  Message  Type  1,  the 
number  of  keying  actions  required  was  26,  so  the  time  for 
each  subject  to  enter  the  data  for  this  combination  was 
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divided  by  26  to  obtain  the  normalized  times.  These  times 
appear  in  App.  0. 

(2)  Accuracy.  The  accuracy  scores  were  norm¬ 
alized  in  a  similar  manner;  however,  the  normalizing  factors 
were  the  reciprocals  of  the  number  of  complete  data  elements 
to  be  entered  in  each  format  for  each  message  type.  These 
normalized  observations  appear  in  App.  D. 

c.  Organization  of  Data 

In  order  to  minimize  the  effort  necessary  for 
performing  the  ANOV  for  both  speed  and  accuracy#  the  data 
in  App.  D  was  rearranged  into  interaction  tables  as  shown 
in  App.  E  [1] . 

d.  Computations  and  ANOV  TaMes 

The  computations  necessary  for  obtaining  the 
results  to  be  entered  in  the  ANOV  tables  were  accomplished 
vising  the  set  of  equations  summarized  in  Ref.  1.  The 
general  form  of  an  ANOV  table  is  discussed  in  Ref.  1,  also; 
therefore,  it  will  not  be  presented  here. 

2.  Results 

a.  ANOV  for  Speed  of  Data  Entry 

The  ANOV  table  obtained  for  the  dependent  vari¬ 
able  speed  in  this  test  is  given  in  Table  VII.  The  effects 
of  the  different  format  levels  and  message  types  are  sig¬ 
nificant  at  a  =  0.05  as  are  the  interactions  of  format  with 
both  message  type  and  subject.  The  acceptance  or  rejection 
of  the  hypotheses  concerning  speed  of  data  entry  is  given 
below: 
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Hol 

Rejected 

Ho3 

Accepted 

Ho5 

Rejected 

Ho7 

Rejected 

H09 

Rejected 

H011 

Accepted 

Ho13 

Accepted 

b.  ANOV  for  Accuracy  of  Data  Entry 

Results  obtained  from  the  ANOV  of  the  dependent 
variable  accuracy  are  displayed  in  Table  VIII.  Significant 
effects  are  exhibited  by  formats,  message  types,  and  their 
interaction  at  a  =  0.05. 

During  the  test,  faulty  design  in  the  program 
for  Message  Type  2  entered  in  Format  1  required  a  data 
element  to  be  carried  over  from  one  line  to  the  next.  The 

i.1. 

margin  for  the  80  character  on  the  grid  screen  was  not 
visible  so  several  subjects  had  difficulty  determining 
when  to  cease  entering  data  on  the  first  line  and  to  con¬ 
tinue  on  the  second.  The  higher  incidence  of  errors  for 
Format  1  and  Message  Type  2  made  an  adjusted  ANOV  for 
accuracy  desirable.  Therefore,  another  ANOV  was  conducted 
with  the  errors  which  were  directly  attributable  to  this 
programming  problem  omitted  from  the  observations.  Table  IX 
displays  the  adjusted  ANOV  results.  As  shown,  format  and 
message  type  effects  are  no  longer  significant  though 
their  interaction  remains  so.  For  the  two  ANOV  for  accuracy 
of  data  entry,  the  acceptance  or  rejection  of  the  hypotheses 


is: 
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■TABLE  VII 


ANOV  TABLE  FOR  SPEED  OF  DATA  ENTRY 


Source 

df 

SS 

MS 

F 

Replications 

1 

.096 

.096 

1.215 

Treatments 

Format  (A) 

3 

2.330 

.777 

9.835* 

Msg.  Type  (B) 

4 

.663 

.166 

2.101 

Subject  (C) 

7 

8.706 

1.244 

15.746* 

AB 

12 

2.552 

.213 

2.696* 

AC 

21 

3.311 

.158 

2.000* 

BC 

28 

1.871 

.067 

.848 

ABC 

84 

5.798 

.069 

.873 

Error 

159 

12.613 

.079 

Total  * 

319 

37.938 

* 

Significant  at  a  = 

0.05. 

All  others 

were  tested  against 

a  =  0.05  and  were  not  significant. 
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TABLE  VIII 


ANOV  TABLE  FOR  ACCURACY  OF  DATA  ENTRY 


Source 

df 

SS 

MS 

F 

Replications 

1 

.00006 

.00006 

.03 

Tr  atments 

Formats  (A) 

3 

.02679 

.00893 

5.02* 

Msg.  Types  (B) 

4 

.03520 

.00880 

4.94* 

Subjects  (C) 

7 

.01167 

.00170 

.96 

AB 

12 

.13266 

.01106 

6.21* 

AC 

21 

.03989 

.00190 

1.07 

BC 

28 

.05343 

.00191 

1.07 

ABC 

84 

.11638 

.00139 

.78 

Error 

159 

.28319 

.00178 

Total 

319 

.69927 

* 

Significant  at  a  = 

• 

in 

o 

• 

o 

All  others  were 

tested 

against 

a  =  0.05  and  were  not  significant. 
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TABLE  IX 


ADJUSTED  ANOV  TABLE  FOR  ACCURACY 
OF  DATA  ENTRY 


Source 

df 

SS 

MS 

F 

Replications 

1 

.00026 

.00026 

.17 

Treatments 

Formats  (A) 

3 

.01105 

.00368 

2.44 

Msg.  Types  (B) 

4 

.00824 

.00206 

1.36 

Subjects  (C) 

7 

.01844 

.00264 

1.75 

AB 

12 

.11210 

.00934 

6.19* 

AC 

21 

.03085 

.00147 

.97 

BC 

28 

.05538 

.00198 

1.31 

ABC 

84 

.11288 

.00134 

.89 

Error 

159 

.24037 

.00151 

Total 

319 

.58957 

*Significant  at  a  = 

0.05. 

All  others  were 

tested 

against 

o  =  0.05  and  were  not  significant. 


Original 

Adjusted 

Ho2 

Rejected 

Accepted 

Ho4 

Rejected 

Accepted 

Ho6 

Accepted 

Accepted 

00 

O 

n 

Rejected 

Rejected 

Hol0 

Accepted 

Accepted 

Ho12 

Accepted 

Accepted 

Ho14 

Accepted 

Accepted 

B.  NONPARAMETRIC  ANALYSIS 
1.  Procedure 

a.  Data  Collection 

Immediately  upon  completion  of  each  test  run 
each  subject  completed  a  Subject  Reaction  Form  on  which  he 
marked  his  evaluation  of  the  ease  of  use  and  suitability 
of  the  format  tested. 

b.  Analysis 

The  Friedman  Test  is  discussed  thoroughly  by 
Conover  and  Kirk  [Refs.  6  and  7].  It  requires  two  assump¬ 
tions  (which  are  considered  valid  for  this  test) :  (1)  each 

subject's  evaluation  of  the  foujr  formats  will  be  independent 
of  every  other  subject's  evaluation,  and  (2)  a  subject's 
evaluations  may  be  ranked  in  some  manner.  The  analysis  for 
both  ease  of  use  and  suitability  is  given  in  App.  F. 


2.  Results 


a.  Ease  of  Use 

The  test  statistic  for  this  analysis  was  15.96 
which/  when  compared  with  the  0.95  quantile  of  a  Chi  Square 
random  variable  with  three  degrees  of  freedom,  is 
significant. 

b.  Suitability 

For  this  analysis,  the  test  statistic  was  11.64 
which  is  also  significant  at  the  0.05  level. 

c.  Hypotheses 

Both  Hq15  and  HQlg  were  rejected  at  the  0.05 
level,  i.e.,  the  subjects  were  not  indifferent  to  the 
formats  as  hypothesized. 
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VI.  CONCLUSION 


A.  PAR7  .IETRIC  ANOV 

1.  Speed  of  Data  Entry 

The  results  of  the  ANOV  for  this  dependent  variable 
clearly  show  that  format  variation  does  affect  speed  of  data 
entry;  thus#  the  primary  purpose  of  this  test  was  success¬ 
fully  accomplished.  Hoi/ever,  due  to  significant  inter¬ 
actions  of  the  formats  with  both  message  types  and  subjects, 
interpretation  of  significant  main  effects  must  be  qualified. 
Figure  3  illustrates  the  type  of  interaction  detected  between 
formats  and  subjects. 


Figure  3.  Example  of  a  Typical  Format-Subject  Interaction. 

It  appeared  that  groups  of  subjects  reacted  in  similar  man¬ 
ners  but  that  each  group  reacted  differently  from  the  other 
groups.  For  example,  subjects  1,2,7,  and  8  all  reacted  to 
the  formats  in  the  same  order,  i.e.,  Formats  4,3,1,  and  2 


(from  fastest  to  slowest),  whereas.  Subjects  4,5,  and  6 
reacted  in  the  order  3, 4, 2,1,  and  for  Subject  3  the  order 
was  2, 3, 1,4.  No  discernible  pattern  of  subject  character¬ 
istics  such  as  I.Q.,  typing  speed,  test  bed  experience,  MOS, 
etc.,  was  evident  within  the  groups.  Though  the  subject 
levels  were  considered  fixed,  it  may  be  concluded  that  no 
single  format  will  be  best  for  all  the  operators  in  a  fielded 
system.  For  any  format  chosen,  under  the  present  concept, 
some  operators  will  not  be  able  to  function  optimally.  This 
implies  that  rigid  selection  and  training  programs  must  be 
instituted  in  order  to  minimize  the  effects  due  to  operator 
variations.  The  format-message  type  interaction  implies 
that  some  message  types  are  inherently  faster  (or  slower) 
when  entered  in  certain  formats. 

2.  Accuracy  of  Data  Entry 

The  original  ANOV  clearly  rejected  the  hypothesis 
that  neither  format  nor  message  type  would  affect  accuracy 
of  data  entry.  However,  when  ignoring  those  errors  directly 
attributable  to  the  program  error,  an  adjusted  ANOV  resulted 
in  the  acceptance  of  these  hypotheses.  The  results  of  the 
adjusted  ANOV  indicate  that  these  factors  have  little  effect 
on  accuracy  of  data  entry;  however,  their  interaction  is 
significant.  This  implies  that  some  of  the  message  types 
are  inherently  more  accurate  (or  inaccurate)  when  used  with 
certain  formats. 
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B.  NONPARAMETRIC  ANALYSIS 

Both  hypotheses  tested  were  rejected.  The  subjects 
tended  to  consistently  rank  the  formats  so  that  one  or  more 
were  favored  over  the  others.  Viewing  the  totaled  ranks  for 
each  format  one  may  infer  that  the  subjects  tended  to  favor 
Format  3  over  the  others  for  both  categories.  In  both  cases 
the  overall  ranking  of  the  formats  was  3,4,1,  and  2. 

C.  SUMMARY 

1.  Purpose  of  the  Test 

As  discussed  in  Para.  I. D, this  test  was  designed  to 
evaluate  differences  in  operator  data  entry  speed  and 
accuracy  resulting  from  the  use  of  alternative  message  for¬ 
mats  when  used  with  differing  types  of  I/O  data.  The  test 
was  not  designed  to  determine  which  format  would  be  best  but 
to  determine  whether  or  not  different  formats  would  affect 
the  dependent  variables.  To  this  extent,  the  purposes  of 
this  test  were  accomplished.  Different  formats  do  affect 
the  speed  of  data  entry  but  do  not  affect  accuracy 
appreciably. 

2.  Inferences 

Due  to  the  format-subject  interaction,  no  one  format 
can  be  inferred  with  certainty  to  be  superior  to  the  others. 
However,  the  mean  speed  for  Format  3  was  the  smallest  of 
the  four  formats  tested  and  its  error  rate  was  very  low. 

It  received  no  rank  below  two  for  ease  of  use  and  was  con¬ 
sidered  first  by  four  subjects  and  tied  for  first  by  one. 
Concerning  operational  suitability.  Format  3  was  ranked 
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first  by  four  subjects  and  tied  for  first  by  one.  It  war 
definitely  preferred  over  the  other  three  formats  tested  by 
the  majority  of  the  subjects  even  though  it  was  not  fastest 
for  several.  Thus,  Format  3  may  be  inferred,  with  reserva¬ 
tions,  to  be  superior  for  vise  in  a  fielded  system. 

D.  RECOMMENDATIONS 

As  shown,  the  evidence  tended  to  place  Format  3  over 
the  other  three  formats  for  use  in  the  MIFASS.  Further 
testing  should  be  contemplated  to  determine  the  reasons 
for  subject  and  message  type  effects.  If  subject  effects 
can  be  overcome  through  selection  and  training  of  operators, 
then  further  development  can  be  directed  toward  determining 
optimum  formats  for  the  message  types  to  be  used.  This 
study  can  be  integrated  into  the  ongoing  development  of  the 
entire  system  if  time  does  not  permit  other  tests  to  be 
conducted  alone  and  proper  precautions  are  taken  to  prevent 
confounding  of  variables.  Another  aspect  to  be  approached 
is  the  weighting  of  message  types  with  regard  to  their 
importance  to  the  success  of  the  system.  If  a  particular 
message  type  is  crucial  to  the  success  of  the  system,  it 
should  have  more  weight  when  optimal  formats  are  being 
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APPENDIX  A 


FORMATS  USED  IN  THIS  TEST 
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APPENDIX  B:  MESSAGE  INPUT  STIMULI 
MESSAGE  TYPE  1 


Field  Name 

Abbreviation 

Requested  Entry 

Check  Fire  Msg. 

CHKFIM; 

From 

FR: 

AR61 

To 

TO: 

BC30 

Target  Number 

TGT  NO: 

BD2121 

Request  Number 

RQNO: 

162310 

Date  Time  Group 

DTG: 

XXXXXXXX 

MESSAGE  TYPE  2 

Field  Name 

Abbreviation 

Requested  Entry 

End  of  Mission 

EOMBDM; 

Battle  Damage 

From 

FR: 

AA31 

To 

TO: 

BA20 

Target  Number 

TGT  NO: 

BB0100 

Request  Number 

RQNO: 

210000 

Casualties 

CAS: 

00025 

%  Destroyed 

%DEST: 

99 

Enemy  Posture 

ENMY  PSTR: 

Bum 

Date  Time  Group 

DTG: 

XXXXXXXX 
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MESSAGE  TYPE  3 


Field  Name _ Abbreviation _ Requested  Entry 


Target  List  Input 

TGTLST ; 

From 

FR: 

CA35 

To 

TO: 

AC40 

UTM  Location 

GRID: 

DE64022610 

Altitude 

ALT: 

00045 

Target  Type/Subtype 

TTYP : 

CEN  /BN 

Size 

SIZE: 

100X100 

Attitude 

ATT: 

08 

Degree  of  Protection 

DOP: 

HV 

Quantity 

QTY: 

1 

Target  Priority 

PRI: 

1 

Target  Status 

STATUS : 

CONF 

Date  Time  Group 

DTG: 

XXXXXXXX 

MESSAGE  TYPE  4 


Fuze 

FZ: 

Q 

Distribution 

DISTRB: 

LAT 

Method  of  Fire 

MOF: 

co:it 

Method  of  Control 

MOC: 

FFE 

Observer  Target 
Direction 

OT  DIR: 

018 

Request  Number 

RQNO: 

012116 

Date  Time  Group 

DTG: 

xxxxxxxx 
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MESSAGE  TYPE  5 


Field  Name _ Abbreviation _ Requested  Entry 


Fire  Order  Universal 

FORDRM; 

From 

FR: 

EC10 

To 

TO: 

AE00 

Weapon  Category 

WCAT: 

ARTY 

Target  Number 

TGT  NO: 

BR2361 

Request  Number 

RQNO: 

390061 

UTM  Coordinates 

GRID: 

DE65022500 

Altitude 

ALT: 

00020 

Method  of  Control 

MOC: 

ADJ 

Method  of  Engagement 

METH  ENG: 

MRK/HA 

Distribution  of  Fire 

DISTRB: 

ZN 

Time  to  Open  Fire 

OPN  FIR: 

0600 

Adjusting  Unit 

UADJ : 

G/3/10 

Number  of  Adjusting 
Rounds 

RDS: 

00003 

Adjusting  Ordnance 

ORD: 

HE 

Adjusting  Fuze 

FZ: 

Q 

Adjusting  Lot 

LOT: 

XY 

Adjusting  Charge 

CHG: 

4 

Adjusting  Angle 

ANGLE: 

26 

Adjusting  Time  on 
Target 

TOT: 

0640 

Fire  for  Effect  Unit 

UFFE: 

H/3/10 

FFE  No.  of  Rounds 

RDS: 

00004 

FFE  Ordnance 

ORD: 

I  CM 
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FFE  Fuze 

FZ: 

TI 

FFE  Lot 

LOT: 

TW 

FFE  Charge 

CHG: 

3 

FFE  Aagle 

ANGLE: 

16 

FFE  Gun  Target  Line 

GTAZ: 

3200 

FFE  Unit 

UFFE: 

1/3/10 

FFE  No.  of  Rounds 

/ 

RDS: 

00006 

FFE  Ordnance 

ORD: 

SMKRED 

FFE  Fuze 

FZ: 

TI 

FFE  Lot 

LOT: 

YX 

FFE  Charge 

CHG: 

5 

FFE  Gun  Target  Line 

GTAZ: 

1800 

FFE  Unit 

UFFE: 

K/3/10 

FFE  No.  of  Rounds 

RDS: 

00002 

FFE  Ordnance 

ORD: 

WP 

FFE  Fuze 

FZ: 

VT 

FFE  Lot 

LOT: 

TE 

FFE  Charge 

CHG: 

2 

FFE  Angle 

ANGLE: 

24 

FFE  Gun  Target  Line 

GTAZ: 

4000 

Date  Time  Group 

DTG: 

xxxxxxxx 
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62 


Illllllillllllll 


I  cm  r->  l  co  I  co 


ACCURACY  OF  DATA  ENTRY 


64 


Message  Type 


APPENDIX  D:  NORMALIZED  TEST  DATA 


r-l  T*  n-  o  tn 

n  in  ri  in 

cm  co  o  co  h 


cocoocivor'-rHci 

COinmOOCOCMVOrH 

COOVOCOmCMOO 


ID'TIDHinCOGIIO 

fniaoiniiofviij'Vfl 

cior"iNino<ofM 


'finduihoiinin 

(nhiniohoiMri 

®<nc\iYir'iooH 


r^coocoovini-Hio 

coonTfoihhi.) 

oocvococmocoo 
•  ••••«** 
»H  rH  rH  rH  rH  i-H  «H 


Is  in  eo 
vo  co  cn 
co  o  oo 


f'-  C"  vo 
in  M  oo 

rH  CM  © 


h  n  in 
co  ov  vo 
o  o  o 


rH  r-  ov  CO 
00  rH  O  in 

rH  r-  o  in 


N  O'  O'  O 
CM  VO  rH  in 
rH  VO  CM  C\ 


VO  rH  O'  O 
O  CM  OV  CO 
N  'f  t'  vf 


rH  VO  C" 

n-  rH  -vj< 

00  O  O 


in  in  o  o 

cm  o  o 

•o1  o  oo  in 


in  in  o  in 
cm  r-  o  H 
ov  c\  o  ^ 


o  vo  cm  n- 
o  in  co  ov 
in  H  o  o 


in  o  o  o 

t"-  o  o  o 

CM  O  rH  n- 


rH  I — loo 
vo  c  in  in 
in  h  ^  ^ 


CM  m  CO  CM  O'  VO  00 

rH  VO  CO  O  CM  OV  O 

■O'  O'  CO  O'  00  rH  VO 

•  •  •  •  •  i  •  • 

rH  rH  rH  rH  rH  ( — 1  rH  r-H 


vo  in  co  in  oo  co 

VO  C'  co  m  H  rH 

00  o  M1  Ol  M1  CO 


o'vovooocor-'oco 

oorHvocMoovinm 

CTlOCMlOCMr'-CMO’ 


•M1  CM  00  o  00  I"  in 

inn-rHoovrHr^o 
Hoor^o^r-  ocM 

•  •  «  •  Cl  •  i  • 


in  o  in 
cm  m  cm 
o>  o  o 


*3>  o  oo 

r^-  in  o  co 

in  vo  o  o 


CM  rH  rH 


co  I  ci  in  o 

VO  rH  o 
r-  rH  o 


1 


ov  oo  in  co  vo  cm  m 

VO  O  rH  o  •M’  VO  rH 

O'  00  rH  CO  00  cn  rH 


o  in  m  in  in 

o  t"  CM  CM  CM 

CO  rH  Cl  l"  rH 


r"  oi  oo 
o  in  o' 

O'  CM  rH 


VO  rH  co 

cm  oo  <n  o 
ci  m  in  vf 


941  1.598  1.631  1.509 


451  1.262  1.289  1.009 


ACCURACY  OP  DATA  ENTRY 


143  0  0  >24g  0  0  .023 


ACCURACY  OF  DATA  ENTRY 


o  ©  o  ol  o  ol  o  I  o 


<n<nocoo<no 
00  o 


n  ro 

oo<nooo<no 

o  o 


©  goo  00  00 


a) 

ft 

>1  <0 

Eh 


00  © 

00000000 


CO  CO  00 

If 

00000000 


o  oi  ol  00000 


ol  ol  ol  ol  o  I  o I  o  I  o 


OOOOIOOOO  ol  ol  ol  ol  ol  ol  ol  o 


co  co  •  vo 

00  00  vo 

OOOOOOO  OOO  OOr-HOO 


OOOOOOOO  OOOOOOOO 


OOOOIOOOO  H  |  o  |  o  |  o  |  o  |  o  |o  jo 


ffl  Ol  O  Ol  O  O  O  O  O  O  O  O  OIO  O  O  O 


<|  ■  0  0  N  O  0  0  o  p  ol  O  I  O  I  O  |  o  lo  |o  |o 


■P - ! - - 1 - .p - 

<a  <o 

e  e 

P«HCNm'3,U-)V£>r'C0  prH<NfO,'3,invOl^OO 

o  o 

Cm  Cm 
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APPENDIX  E:  INTERACTION  TABLES 
SPEED  OF  DATA  ENTRY 


AXBXC 


1 

2 

3 

n 

5 

a 

2 

3 

4 

5 

B 

1.308 

2.400 

1.758 

1.870 

1.723 

2.259 

2.079 

1.880 

2.449 

2.024 

2 

1.423 

2.000 

2.919 

2.039 

1.764 

3.963 

1.922 

1.956 

2.858 

2.262 

3 

1.808 

1.800 

3.077 

2.747 

2.231 

1.519 

1.743 

2.132 

2.398 

1.984 

4 

2.153 

2.915 

2.343 

2.600 

2.610 

2.259 

2.300 

2.403 

2.608 

2.373 

5 

2.616 

3.836 

2.799 

3.342 

2.940 

4.111 

2.124 

2.636 

3.245 

3.384 

6 

1.615 

2.171 

2.231 

2.727 

2.271 

2.888 

1.643 

1.752 

1.990 

2.226 

7 

2.077 

2.550 

2.403 

2.374 

2.371 

2.852 

2.802 

2.668 

3.038 

3.508 

8 

2.115 

3.150 

2.133 

2.381 

_____ 

2.174 

_ 

2.555 

4.330 

2.766 

3.539 

3.140 

1  1.770  1.436  1.838  2.310  1.888 


2  1.50011.513  2.370  1.584  1.808 


3  2.616  1.436  2.322  1.925  2.380 


4  1.923  1.769  2.387  2.136  2.084 


5  2.116  2.411 


2.523  2.684 


6  1.423  1.436  1.710  1.800  1.930 


2.153  2.410  2.564  2.881  2.726 


8  2.730  2.539  2.565  2.577  2.763 


3 

4 

5 

1.542 

1.617 

2.049 

1.738 

1.713 

2.137 

2.694 

2.691 

2.109 

2.177 

2.363 

2.291 

3.242 

2.397 

2.733 

2.064 

1.894 

2.224 

2.156 

2.543 

2.319 

2.215 

2.713 

2.298 

70 


u  iii 


AXC 


BXC 


72 


8  19.5 

1  Rii 
i=l  13 


30 


13 


17.5 


x2  =  8T4TW  [380-25  +  900  +  169  +  306.25]  -  3(8)  (5) 


=  11.64* 


X0,05,3 


=  7.82 
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